1/1 


AD-A146  215 
UNCLASSIFIED 


A  TWO-SEGMENT  APPROXIMATION  ALGORITHM  FOR  SEPARABLE 
CONVEX  PROGRAMMING  WI.  .  (U)  TEXAS  UNIV  AT  AUSTIN  CENTER 
FOR  CVBERNETIC  STUDIES  I  ALI  ET  AL.  JAN  84  CCS-RR-476 
N00014-81-C-0226  F/G  12/1 


NL 


AD- A 146  215 


/ 

L 


/ 


Research  Report  CCS  476 

A  TWO-SEGMENT  APPROXIMATION  ALGORITHM  FOR 
SEPARABLE  CONVEX  PROGRAMMING  WITH 
LINEAR  CONSTRAINTS 

t>y 

I.  Ali 
A.  Charnes 
T.  Song 


CENTER  FOR 
CYBERNETIC 
STUDIES 

The  University  of  Texas 
Austin, Texas  78712 


84  09 


27  019 


if) 


VA 


Research  Report  CCS  476 

A  TWO-SEGMENT  APPROXIMATION  ALGORITHM  FOR 
SEPARABLE  CONVEX  PROGRAMMING  WITH 
LINEAR  CONSTRAINTS 


I.  All 
A.  Charnes 
T.  Song 


January  1984 


k  OCT  2  1984  ;; 


This  research  was  supported  in  part  by  ONR  Contracts  N00014-81-C-0236  and 
N000I4-82-K-0295  and  USARI  Contract  MDA-903-S3-K-0312  with  the  Center  for 
Cybernetic  Studies,  The  University  of  Texas  at  Austin.  Reproduction  in  whole 
or  in  part  is  permitted  for  any  purpose  of  the  United  States  Government. 


CENTER  FOR  CYBERNETIC  STUDIES 

A.  Charnes,  Director 
Graduate  School  of  Business  4.138 
The  University  of  Texas  at  Austin 
Austin,  Texas  78712 
(512)  471-1821 


A  Two-Seqment  Approximation  Algorithm  for 


Separable  Convex  Programming  wit 


Linear  Constraints 


I.  Ali,  A.  Charnes,  T.  Song 
Abstract 

We  discuss^a  new  algorithm  for  the  separable  convex  programming  with 
linear  constraints.  This  is  based  on  the  approximation  of  the  objective 
function  by  at  most  two  linear  pieces  in  the  neighborhood  of  the  current 
feasible  solution.  The  two  segments  will  be  adaptively  defined  rather  than 
predecided  fixed  grids.  If,  furthermore,  the  objective  function  is  differ¬ 
entiable,  and  one  introduces  a  non-Archimedean  infinitesimal,  the  algorithm 
generates  a  sequence  of  feasible  solutions  every  cluster  point  of  which  is 
an  optimal  solution.  Computational  tests  on  the  problem  with  up  to  196 
non-linear  variables  is  presented.  (" 
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A  TWO-SEGMENT  APPROXIMATION  ALGORITHM  FOR  SEPARABLE  CONVEX 
PROGRAMMING  WITH  LINEAR  CONSTRAINTS 

by 

I.  All,  A.  Charnes,  T.  Song 

1.  Introduction 

In  1954,  Charnes  and  Lemke,  in  order  to  bring  to  bear  the  computational 
power  of  linear  programming  methods  for  computation  of  non-linear  convex 
programming,  developed  a  new  linear  programming  algorithm  which  obviated 
the  need  to  treat  explicitly  as  additional  constraints  those  which  were 
upper  bounds  on  individual  variables  (see  [1]).  Specifically,  the  reduction 
of  the  general  convex  quadratic  programming  problem  to  such  terms  was 
done  by  first  introducing  new  variables  and  constraints  in  terms  of  which 
the  objective  function  was  separable  in  the  new  variables.  Then  one  approx¬ 
imated  the  separable  functions  by  piece-wise  linear  ones,  each  piece  of 
which  corresponded  to  an  additional  variable,  individually  upper  bounded,  whose 
sums  would  be  such  a  new  variable.  This  procedure,  although  clumsy  and  sometimes 
slow  in  computation  to  its  optimum,  an  approximation  to  the  optimum  desired,  was 
nevertheless  stable  and  was  generalized  to  a  large  class  of  differentiable 
convex  programs  with  convex  constraints  by  Griffith  and  Stewart  by  use  of 
first-order  Taylor  series  expansions  of  the  objective  and  constraint  functions. 
Successive  refinement  of  these  separable  programming  problems  was  employed  to 
achieve  better  approximations  to  the  exact  optimal  solution.  See  also  Charnes  and 
Cooper  in  [2]. 

In  1959,  Charnes  and  Lemke  developed  a  non-linear  analog  of  the 
simplex  algorithm  which  obviated  the  need  to  employ  such  successive  linear 
programming  approach.  This  method  was  never  computationally  tested  since 
Lemke  and  Howson  were  unable  to  debug  their  computer  code  in  what  time 
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remained  from  their  major  effort  of  developing  the  bimatrix  equilibrium 
solution  and  Lemke's  complementarity  theory. 

In  the  intervening  years,  non-linear  programming  computation  has 
focused  on  either  constrained  gradient  methods  or  the  parametric  unconstrained 
methods  (of  Fiacco  and  McCormick)  which,  for  computational  stability,  have 
required  such  complications  as  updating  Hessian  matrices  in  Newton-Raphson 
methods.  Recently,  efforts  have  swung  back  toward  the  original  procedures 
now  designated  as  SLP,  "Successive  Linear  Programming."  But  these  procedures 
themselves  can  be  extremely  slow  and  practically  unusable  if  implemented  by 
existing  algorithms  which  are  otherwise  perfectly  suitable  for  the  most 
frequent  usages  of  these  algorithms.  Thus,  for  example,  Charnes  and  Phillips 
in  solving  a  separable  convex  programming  problem  with  pure  network  constraints 
via  an  approximating  pure  network  problem  with  multiple  arcs  between  the  same 
nodes,  discovered  that  the  solution  time  for  these  with  PNET,  the  Klingman, 
Karney,  Glover,  and  Stutz  world- preeminent  primal  network  code,  or  with  Barr's 
world-renowned  SUPERK  code,  the  world's  preeminent  out-of-kilter  code,  was 
several  orders  of  magnitude  slower  in  solution  than  anticipated  for  usual 
network  problems  of  the  same  size.  This  difficulty  was  overcome  with  the 
invention  of  a  new  algorithm  which  required  no  more  than  three  halves  the 
computation  time  to  be  anticipated  from  PNET  on  ordinary  problems. 

The  key  efforts  toward  effective  methods  for  non-linear  convex  programming 
problems  ought  therefore  to  concentrate  on  development  of  new  algorithms  for 
the  special  structures  and  inter-relations  between  linear  programming  problems 
arising  in  a  sequential  linear  programming  approach  to  their  solution.  In 
this  paper,  we  consider  convex  separable  programs  with  linear  constraints 
and  build  on  previous  work  by  R.R.  Meyer  [4]  to  develop  a  new  algorithm  which 
appears  thus  far  to  have  superior  convergence  properties  (speed)  as  well  as 
conveniences  of  representation  than  does  his.  In  particular,  special  use  is 
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made  of  adaptive  bounding  and  grids  rather  than  the  fixed  a  priori  grids  he 
employs.  Convergence  properties  of  the  new  method  are  established  and  some 
examples  are  given  of  its  performance. 

Thus  in  this  paper  we  present  a  two  segment  approximation  algorithm  with 
adaptively  defined  rather  than  predecided  segments.  If,  further,  one  intro¬ 
duces  a  non-Archimedean  infinitesimal,  the  algorithm  generates  a  sequence  of  feasibl 
solutions  every  cluster  point  of  which  is  an  optimal  solution. 

Consider  the  following  program. 

P 

min  2  =  £f- (x.)  +  gTy 
i=l  1  1 

s.t. 

(P)  Ax  +  Dy  =  b 

1  <  x  <  u 

y  <  y  y 

where  f.(*)  is  a  convex  function  defined  on  an  interval  containing  [K,u^]; 
x,  1,  u  are  p-vectors  with  1^  <  ui  ,  i  =  l,...,p  ;  y,  g,  y,  y  are  q  vectors 
and  <  y^,  i  =  l,...,q;  A,  D  are  m  x  p  and  m  x  q  matrices  respectively. 

The  basic  idea  is  that  in  a  neighborhood  1  <  x  ^  u  ,  y  y  <  y  of  the 
current  feasible  solution  (m,y),  each  f.(x^)  in  program  (P)  will  be  approximated 
by  at  most  two  linear  pieces  as 
in  the  figure.  The  currently 
chosen  bounds  of  this  neighborhood 
are  called  "artificial  bounds." 

Using  this  approximation  of  the 
object  function  together  with 
these  artificial  bounds,  we  solve 
the  linear  programming  problem. 


If  the  current  point  is  optimal,  or,  if  the  optimal  solution  of  the 
linear  program  has  no  component  at  an  artificial  bound,  we  halve  the  size  of 
the  artificially  bounded  neighborhood,  and,  based  on  this  optimal  solution 
we  construct  another  approximating  program.  Otherwise,  we  keep  the  approx¬ 
imation  unchanged  for  those  variables  that  are  not  at  the  artificial  bound 
whereas  for  those  variables  at  artificial  bounds  we  construct  a  new  approx¬ 
imation  on  the  current  solution  value  with  the  same  size  interval  as  before. 

We  then  solve  the  new  program.  These  procedures  are  repeated  until  specific 
criteria  to  be  detailed  later  are  satisfied. 

In  our  procedures,  x^  will  be  split  into  three  parts:  a  constant  part 
w.j ,  a  decreasable  part  and  an  increasable  part  and  ni  correspond 

to  the  left  segment  and  right  segment  respectively.  is  initially  set  at 
its  upper  bound  and  is  a  non-basic  variable.  It  can  only  be  decreased.  Its 
lower  bound  0  corresponds  to  an  artificial  bound.  In  contrast,  is  initially 
set  at  its  lower  bound  and  may  be  a  basic  variable  if  x^  is  a  basic  variable  of 
an  approximating  program.  Its  upper  bound  corresponds  to  an  artificial  bound. 
Thus,  there  is  a  basic  feasible  solution  available  when  one  starts  a  new 
approximating  program. 

In  the  next  section,  we  will  give  the  detailed  transformations  and  the 
algorithms.  In  section  3,  we  will  prove  that  usually  the  algorithm  will 
terminate  in  a  finite  number  of  iterations.  If  one  introduces  a  non-Archimedean 
infinitesimal  and  if  the  objective  function  is  differentiable,  then  the  algorithm 
will  produce  an  infinite  sequence  of  feasible  solutions  every  cluster  point 
of  which  is  an  optimal  solution  of  the  original  program.  In  section  4,  some 
computational  results  based  on  network  constraints  will  be  presented. 


We  introduce  two  transformations.  The  first  one,  based  on  a 
feasible  solution  of  program  (P),  creates  a  two  segment  pi ecewise-1 inear 


approximation  program  with  a  certain  neighborhood  size.  The  second  one  only 

changes  part  of  the  approximation  problem  corresponding  to  a  variable  at  its 

artificial  bound  in  the  optimal  basic  solution  of  the  approximating  problem. 

Suppose  (x,y)  is  a  feasible  solution  of  (P)and  5  is  a  positive  real 

number,  usually  less  than  min  (u-  -  1-).  The  transformation  I  gives  another 

i  1  1 

program  P(x,y,<5)  as  follows: 


P(x,y,S) 


P 

min  z  =  d^C  +  e^n  +  g^y  +  T^f .  (w. ) 

i=l  1  1 

s.t. 

A£  +  An  +  Dy  =  b  -  Aw 


0  <  £  <  a 


0  <  n  ^  6 

i  <  y  <  y 


where  a.  =  min  (6,  x^  -  1^)  ,  ^  =  min  (6,  ui  -  x.. )  , 


Wi  =  x.  -  a.  , 

-M 

di  =  (fi(xi)  -  fi(wi))/oi 


,  if  ai  =  0 
,  if  ai  >  0 


(  M  .  if  Si  - 

1  "  (  (f i (xi+^i )  -  fi(xi))/6i  ,  if  Bi  > 


=  0 
0 


for  i  =  l,...,p  and  M  is  a  large  positive  number. 


Clearly,  if  (x,y)  is  a  feasible  solution  of  (P),  then  (ot,0,y)  must  be  a 
feasible  solution  of  P(x,y,6).  Conversely,  if  (£,n,y)  is  a  feasible  solution  of 
P(x,y,<5)  then  (w+£+n,y)  is  a  feasible  solution  of  (P).  As  a  matter  of  convenience 
we  say  xi  is  a  basic  variable  if  either  ^  or  is  a  basic  variable.  Also,  if 


x^  is  a  basic  variable,  after  transformation  I  would  be  counted  as  a  basic 
variable  even  if  it  is  at  its  lower  bound. 

AAA 

Suppose  (C.n»y)  is  an  optimal  basic  solution  of  P(x,y,(5).  Let 


J1  =  {i  :  =  0  ,  ai  >  0} 

A 

J2  =  {i  :  ni  =  Bi  >  0} 


Definition:  The  optimal  solution  (C,n,y)  of  P(x,y,6)  is  called  "regular"  if  and 


only  if  n-  >  0 


£ .  =  a. 

1 


Since  f.j(x.j)  is  convex,  d^  <  e^.  Thereby,  a  perturbation  can  guarantee 
that  the  simplex  algorithm  will  give  a  regular  optimal  solution.  For  a  regular 
optimal  solution  =  <f>,  and  the  following  transformation  II  will  change 

A  A  A  A  A 

P(x,y,6)  into  another  program  P  (x,y,6)  of  the  same  type,  where  x  =  w  +  £  +  n. 


mm  z 


v 

dT£  +  eTn  +  <$Ty  +  £)f-(w. ) 

i=l  1  1 


P(x,y,6) 


+  An  +  Dy  =  b  -  Aw 
0  <  £  <  a 


0  <  n  <  B 


y<y<y 


where 


min  (x. 

6 


min  (u. 


s  .  I  5i  . 

Wi  i  x.  -  a. 


li  ,  6) 


x.  ,  6) 


i  i  jxuj2 
i  c  j 
i  €  J2 

i  t  JXUJ2 
i  € 

i  e  J2 

i  t  JjUJ2 
i  €  J,UJo 


.  i  t  JjUJg 


[fi(xi)  -  <ri(w.)]/ai 


i  e  and  a..  =  0 
otherwise. 


ei  ,  i  t  JjUJ2 

M  ,  i  e  J,  and  0. 
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[fi(xi  +  0^  -  1ri (xi )]/Bi  ,  otherwise. 


The  variables  {x^  :  i  are  said  to  be  at  artificial  bounds. 
Actually  the  difference  between  problems  P(x,y,6)  and  P(x,y,6)  is  that  the 
approximation  of  the  objective  function  for  artificial  bound  variables  has 
different  two  piece  segments.  Next,  set 


ai  ,  i  £ 

A 

,  otherwise. 


0  ,  if 

A 

n.j  ,  otherwise 


It  is  easy  to  see  that  (S',n'>y)  is  a  basic  feasible  solution  of 

A  /\ 

P(x,y,6);  and  following  lemma  is  true. 

Lemma  1.  Transformation  II  does  not  change  the  corresponding  objective 
function  value. 

AAA 

Proof:  If  ieJj^  then  *  0  ,  =  0  ,  x..  =  w^ 

f,(w,)  ♦  d1Si  +  e,n,  ■  V»1> 

■w 

=  f,(B,)  +  a,i, 

-  *  ai«i  *  Vi  • 

A  a 

For  i€J2,  there  are  ^  =  c^.  ,  ^  =  6^ 

V"i>  *  Vi  +  ei^i  ■  fi<"i  *  “1  *  V 

’  fi!*1> 

=  f,(w.)  +  a.F!  +  e,n- 
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For  other  variables,  the  corresponding  part  of  the  objective  function 


remains  the  same  as  before. 

■ 

Algorithm 

Step  0:  Initialization.  Starting  with  any  usual  linear  programming 
software,  one  can  get  a  basic  feasible  solution  (x^,y°)  of 
the  program  P,  or  show  that  the  program  P  has  no  feasible 
solution. 

Take  e  >  0  and  0  <  6  <  min  (u^  -  1 . )  where  e  may  be  a  non- 
Archimedean  infinitesimal.  The  transformation  I  gives  a 
program  P(x^,y°,6^)  denoted  by  P1  ,  and  its  basic  feasible 
solution  by  (C°,n°,y°).  Set  k  =  1. 

Step  1:  If  Uk-l,nk-l,yk-l)  solves  the  program  Pk,  set  6k  =  i6k-*, 
(£k,nk,yk)  =  (lk-1,nk-1,yk“1)  and  go  to  Step  3.  If  not,  go 
to  Step  2. 

1/ 

Step  2:  Suppose  that  by  solving  program  P  ,  one  gets  a  regular  optimal 

solution  (5k,nk,yk).  Let 

,k  _  ,-k  „  k 

{i  •  C-j  ~  0  ,  ex.-  >  0}  , 

k  k  k 

=  (i  :  n-  =  B*  >  0}  . 

If  JkUJ2  t  4>  »  go  to  Step  4.  Otherwise  set  6k  =  i6k_1  and  go 
to  Step  3. 

Step  3:  If  6k  <  e  ,  terminate;  otherwise,  set  xk  =  £k  +  nk  +  wk. 

k  k  k  k+1 

By  transformation  I,  one  gets  program  P(x  ,y  ,d  )  denoted  P 

-  k  -  k  k 

and  its  feasible  basic  solution  (£;  ,n  ,y  ).  Go  to  Step  1. 

Qi.n  h  p  i  r k  r k - 1  k  _k  k  k 

Step  4:  Set  6=5  ,  x  =  £  +  n  +  w 

k  k  k 

By  transformation  II,  one  gets  program  P(x  ,y  ,6  )  denoted 

k+1  _  1/  _  k  _  k 

P  and  its  basic  feasible  solution  (£  ,n  ,y  ).  Go  to  Step  1. 


k  k 

If  we  denote  the  optimal  objective  function  value  of  program  P  as  z  , 

following  lemma  holds: 

k 

Lemma  2.  {z  }  is  a  non- increasing  sequence. 

Proof:  Clearly  it  is  sufficient  to  prove  that  transformations  I  and  II 
do  not  increase  the  corresponding  objective  function  value.  By  Lemma  1,  we 

need  only  to  prove  the  assertion  holds  in  the  case  of  transformation  I. 

k  k  k 

If  0  <  £.j  <  a.j  ,  by  regularity  of  solutions,  =  0.  Hence 

By  convexity  of  f .  (•)» 


fX>  -  V"i>  ,  fX +  4> 

ik  '  ak 

S1  1 


V"1> 


Therefore 


ff(xk)  <  f,(wk)  ♦  dk5k 
■ 


On  the  other  hand,  by  transformation  I 


f^O +  di+1  "4 =  vxi} 


-k  _  n 

ni  =  0 


f  /  k+l\  x  Jk+1  ?k  x  „k  +  l  -  k  ^  *  /.  ,k  N  ,  Jkrk  .  k  k 

fi<wi  >  +  +  n,  <  f,(w,)  +  d,?,  + 


k  k  k  k 

By  a  similar  argument,  for  the  case  0  <  <  3^  ,  (£..  =  a^),  the  above 


inequality  also  holds. 


k  k  k  k 

For  =  0  ,  or  =  0  ,  using  the  same  argument  as  in  lemma  1 
it  can  be  proved  that  the  above  inequality  becomes  an  equality.  In  conclusion, 
we  have 


2k+1  ^  ik+1 


i=l 


(wk+1) 


*  dk*l  jk  +  ek+lT  fik  +  gTyk 


2>i<wi> 
i=l  1  1 


♦  dkV  ♦  ekTnk  ♦  gTyk 


3.  Convergence 

In  order  to  explore  the  convergence  properties  of  the  algorithm,  we  need 
the  following  lemma. 

k 

Lemma  3.  There  are  only  a  finite  number  of  6  having  the  same  value. 

k+1  k 

Proof:  Noting  that  6  =  6  if  and  only  if  one  is  at  step  4  and 

k+1  k 

otherwise  6  <  6  ,  it  is  sufficient  to  prove  that  the  interation  between 

step  2  and  step  4  with  the  same  value  6  can  be  repeated  only  finitely  often. 
k-1  k  k+1 

Suppose  that  6  =6=6  .By  lemma  1,  transformation  II  does  not  change 

the  corresponding  objective  function  value.  By  step  1,  we  can  assert  that  the 

k+1  k  _k  -k  k 

optimal  objective  value  z  is  strictly  less  than  z  ;  otherwise  (l  ,n  »y  ) 

k+1  k+1  k 

will  solve  P  and  6  =  H  .  This  is  in  contradiction  to  our  assumption 

,k  ,k+l 

0=0 

So  if  6k  =  6k+1  =  6k+2  =  .  .  .  then  Pk+1,  Pk+2,  .  .  .  would  all  be 
different  approximating  programs.  On  the  other  hand,  the  transformation  II 

k 

with  the  same  6^,  starting  from  some  x  can  generate  as  we  shall  see  only  a 
finite  number  of  different  two  segment  approximations  for  f^(*)-  As  a  matter 

Is 

of  fact,  the  junction  point  of  two  segments  together  with  6  uniquely  decides 

1/ 

the  approximating  program.  But  starting  from  x?,  the  transformation  II  with 

k 

the  same  6  has  only  the  following  finite  number  of  possible  junction  points. 


C. 

•a 


v? 

I 


->  v  v  V  v  *>"v 


.  *  m  m  *r*  ^  '_*  •  *  v  V 


A  -  t/  ,  xk  -  (t,  -  l)6k  . 


k  ,  .k 
x.  +  s.6  , 

]i  »  ]i  +  ^  »  •  •  •  .  ^  +  ri6k  , 

ui  *  ui  “  6  ’  *  *  *  *  U1  "  Vk  * 


k  k  k  k 
,  x*  -  6K,  x*  +  6K  ,  . 


is  i 

xi  -  ’i 


ui  -  ', 


"t,ere  *i ’  L~7~J  ’  s< =  L— nr-J  •  ri =  l—j-J  are  1nte9ers 


This  completes  the  proof. 


From  lemma  3,  it  is  easy  to  obtain  the  following  theorem. 

Theorem  1.  For  any  real  number  e  >  0  the  algorithm  must  be  terminated 
in  a  finite  number  of  iterations. 

Next  if  e  is  taken  to  be  a  non-Archimedean  infinitesimal,  the  stopping 

criteria  would  never  be  satisfied.  Therefore,  the  algorithm  would  produce  an 

infinite  sequence  {xk,yk}.  Since  1  ^  xk  <  u  ,  £  <  yk  <  y  ,  the  {xk,yk}  must 

have  a  cluster  point.  As  we  will  show  next, if  all  f^(*)  are  differentiable 

k  k 

then  every  cluster  point  of  {x  ,y  }  is  an  optimal  solution  of  the  program  P. 

To  do  this  we  first  require  the  following  lemma. 

k  k 

Lemma  4.  If  all  f.(*)  are  differentiable  and  {x  ,y  }  is  the  infinite 
sequence  generated  by  the  procedure,  then  there  exists  a  subsequence  which 
converges  to  an  optimal  solution  of  program  P. 

Is 

Proof;  By  lemma  3,  since  only  a  finite  number  of  6  have  the  same  value, 
there  exists  an  infinite  subsequence  {xnk,ynk}  such  that  6°k+1  =  $6nk  .  Without 
loss  of  generality  we  can  simply  assume  that 

{x°k>  - >  x*  ,  {y°k} - >  y*  ,  s"k - >  0  (k - >  «) 
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If  x|  >  ,  then  for  k  big  enough,  there  are 


and 


xik  >  1i  »  aik  >  0 


d"k  *  Cf 1 <«7k  ♦  »?k) 
=  f;(e"k) 


where  w..k  ^  6^.k  ^  w*?k  + 


VwikO  /ajk 


Hence 


e?k  -  x|  |  <|  ejk  -  x"k  |  +  |  x«?k  .  x*  | 
<  2 6nk  +  |  x”k  .  x*  |  - >  o 


as  k - >  co 


Because  fj(»)  is  a  convex  function  as  well  as  differentiable,  f!(-) 
must  be  continuous  (pp.  246,  corollary  25.5  [5]),  so 


dnk 


->  f;(x|) 


as  k 


->  00. 


Similarly,  if  xt  <  u.  , 


a"k 


->  f'.(x*) 
iv  v 


as  k 
,k. 


->  00. 


k 

Since  the  linear  programs  (P  }  have  only  finitely  many  different  bases, 
some  infinite  subsequence  necessarily  has  the  same  optimal  basis  B.  Without 
loss  of  generality  we  can  assume  that  our  subsequence  {Pnk}  is  one  of  these. 
Denote  the  objective  function  coefficients  of  Pk  corresponding  to  basis 

B  as  Cg  and  (f|(x{),  .  .  .  cjj . gp)  as  Cg. 

If  d^k  is  one  of  Cgk  for  all  k,  then  is  a  basic  variable  and  ak  >  0. 
Using  the  above  argument,  one  can  prove  that  d"k - >  f^(xt).  Similarly, 


if  e^k  is  one  of  Cgk  for  an  k,  we  have  that  e!?k 


->  fj(x*).  In  conclusion 


->  cr 


as  k 


->  00 


m 


Since  6°k+l  =  i6°k  ,  *  4>*  And,  since  B  is  optimal,  we  have 


d"k  <  cjk  B"1  a. 

Id  1 

eik  >  <£k  B'1  «, 


Finally,  for  x*  >  di  ,  f!(xT)  <  CgB‘1ai 
*?<“i  ,  f|(x?)>CBB-1a1 


f;(x*)  -  cbb'  ai  >  0  »  ifxisli 


fi(x*}  ‘  cbb'  ai  <0  •  if  x* =  Ui 


f.'(x*)  -  CgB^a.  =  0  ,  ifl.<x*.<u. 


For  y*  we  have  similar  relations  with  g.  These  relations  demonstrate  that 
(x*,y*)  satisfies  the  Kuhn-Tucker  condition  for  program  P.  Because  P  is  a 
convex  program,  its  Kuhn-Tucker  point  is  also  its  optimal  point. 


Theorem  2.  Under  the  same  assumption  as  in  lemma  4,  every  cluster  point 
k  k 

of  {x  ,y  }  is  an  optimal  solution  of  P. 


Proof:  Let 


fk(Xi> 


f((wk)  ♦  dk(x(  -  wk) 


k  k  k 

,  w*  <  xi  <  wi  +  a* 


f^ (wj  +  aj)  +  eJ(Xj  -  (wj  +  of))  ,  wjf  +  aj  <  xi  < 


,  k  .  k  ,  „k 
<  wi  +  a.  +  Bi 

By  definition  of  the  d^  ,  e*|  ,  recalling  x^  =  ^  ,  it  is  easy 


to  verify  that 


*k,  k*  -  *  /.,k\  jkrk  ,  k  k 
f:(xj  *  t.(w.)  +  d,c.  +  e.n4 


v  .*  -  ^  *  •  /•  -  / -  ^  va^iv  v.'r.\  .v.v  v.v] 


L 

Therefore,  the  optimal  objective  function  value  of  P  is 


2k  -  £fk(x*>  *  «V 

i=l  1  1 


Since  f.(*)  is  a  convex  function  defined  on  an  interval  containing 


[li,ui],  there  exists  a  constant  L.  such  that 

|f,(xj)  -  ^(xf)  I  «L,|x|  -  x* 
(pp.  237,  Thm.  24.7  [5]). 


*  x}  .  x2  e[li,ui] 


If  8  0  ,  ^  >  0  ,  then  ak  >  0  and 


k  .  k. 


k  k  k  k  k  H  (wj  +  ot^ )  ~  ■Pi(wi)  k  k 

|f}(w})  -  ^(xpl  <  |fi(wj)  -  fi(xj)|  +  — — - Ij. - i -J-(xk  -  wf) 


<  2L.|x.  -  wk| 


<  2L.6' 


If  nj  *  0  ,  £k  *  0  ,  then 


|f-(xj)  -  f^xjjl  -  0  <  2L.6k 


By  an  analogous  argument  we  can  show  that  if  >0 
|f-(xk)  -  f.(xk)|  <  2L.6k  . 

Suppose  that  there  is  a  subsequence  {xnk,ynk}  such  that 

(xnk,y"k) - >  (x,y)  as  k  — >  ® 

P  P 

and  that  £f.(x.)  +  gTy  >  . (x*)  +  gTy* 

i=l  11  i=l  1  1 
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‘  (^W**7*)  '  (&VXT>  ♦  ■>) 


L  =  max  {L^} 


There  is  an  N  such  that 


I'  1 

Ef((x?k) ♦  gVk1 ' 
^ 1=1  1  1  \ 

/  p  \ 

(  P  .  T  .  \ 

|(^f(<51>  +9T^J 

'  1 

E^)  ♦  gVk  ] 

<  |  ,  V  kl  >  N 


<  J  ,  V  k2  >  N 


0  <  s"kl  <  T6TT  •  v  kl  > 


0  <  s"k2  <  IFTTl  •  ,k2>H 


Hence 


,"k2|  =  |  z"kl  ♦  gVk1)] 


[(2>i(x?kl) +  9VW)  -  (Ew«V*)]- 


♦  [(Sv*,)  +  gTy  )  -  (£fj(x"k2>  *  gVk2)] 


[(SV^2)  *  gVk2)  -  *]  I 


AAA 

8  '  I  ‘  8 


V  kl  >  N  ,  k2  >  N 


*  V  MiAV 


By  lemma  2,  {zK}  is  a  non- increasing  sequence.  From  lemma  4,  it  has  a 

1/  # 

subsequence  that  has  a  finite  limit.  Therefore  (z  }  itself  converges  to  a  finite 
point.  But  this  contradicts  the  above  inequality.  The  contradiction  shows 
(x,y)  must  be  an  optimal  solution  of  P. 

■ 

Theorem  3.  Under  the  same  assumptions  as  in  Theorem  2,  if  f^(*)  is 
a  strictly  convex  function  and  (x*,y*),  (x,y)  are  both  cluster  points  of 
(xk,yk},  then 


Proof:  By  Theorem  2 

£fj(x»)  ♦gV-Efflx,)  +  gTy 

Since  (ax  +  (1  -  a)x*  ,  ay  +  (1  -  a)y*)  is  also  a  feasible  solution  of 
P  and  f.(-)  are  convex,  (x*,y*)  is  an  optimal  solution  of  P 

+  9^*  *  !>,(<*,  +  U  ’  °0X*)  +  9T(ay  +  (1  -  a)y*) 

<  a  ( Efi(xi)  +  gTy  )  +  (i  -  a)  (Efi(xT)  +  gV) 

=  Efi(x*)  +  9Ty*  • 

Therefore,  there  must  hold, 

f^axj  +  (1  -  a)xt)  =  af^x^)  +  (1  -  ajf^x*)  , 

where  0  <  a  <  1  ,  i  *  1,... ,P 

Since  f^(*)  is  strictly  convex,  we  must  have 


IO 


4.  Computational  Aspects 

We  have  developed  a  FORTRAN  Code  to  implement  the  two  segment  algorithm 
described  in  section  2.  Based  on  the  network  constraints,  some  problems  with 
up  to  169  nonlinear  variables  have  been  computed  on  the  CDC-750/130  at  the  Univer¬ 
sity  of  Texas  at  Austin.  The  computational  results  are  quite  encouraging. 

For  example,  it  took  only  0.08  to  0.14  seconds  CPU  time  to  solve  the  15  non¬ 
linear  variable  problem  A  of  Meyer  in  [4]. 

More  generally,  since  our  algorithm  does  not  receive  an  priori  break 
point  set,  the  preprocessor  work  of  the  linearization  is  no  longer  neccessary. 
Again,  for  ease  of  proof  of  convergence  in  section  3,  we  stipulated  that  if 
a  variable  hit  an  artificial  bound  the  length  of  the  segment  would  remain 
the  sanr.o  whereas  if  not,  the  length  would  be  halved.  Our  computational  ex¬ 
perience,  however,  shows  that  an  alternative  rule  yields  faster  convergence. 

That  is,  whenever  a  variable  hits  an  artificial  bound,  the  length  of  the 
segment  is  expanded  by  a  factor  of  1.25;  otherwise,  the  segment  length  is 
shrunk  by  a  factor  of  0.4.  This  experience  confirms  that  of  Meyer's  in  [4]. 

Problem  1:  This  problem  is  test  problem  A  in  [4]. 

min  2wi<1-<'i)*l/100° 
i=l 

10 

s.t.  X)x.  *  75,000 
i=l 

15 

Y!  x.  =  67,000 
i=6 


0  <  x,  <  u. 


(i  =  1 


15) 
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description  of  the  model  can  be  found  in  [3], 


With  initial  length  of  segment  50,000  and  terminal  length  1,000,  after 


24  iterations  (2.85  per  second  CPU  time)  a  solution  was  obtained.  It  is  of 


course  far  more  accurate  than  that  of  the  approximate  once  and  for  all  piece¬ 


wise  linearization  of  [3]. 


Although  the  two  examples  are  hardly  adequate  to  make  responsible  pro¬ 


jections,  note  further  that  the  increase  from  15  to  169  nonlinear  variables 


involved  a  substantially  less  than  quadratic  increase  in  computer  time. 
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